CS256 Fall 2017Practice Midterm 2

Studying for one of my tests does involve some memorization. I believe this is an important skill. Often people waste a lot of time and fail to remember the things they are trying to memorize. Please use a technique that has been shown to work such as the method of loci. Other memorization techniques can be found off the Wiki Page for Moonwalking with Einstein. Given this, to study for the midterm I would suggest you:

• Know how to do (by heart) all the practice problems.
• Go over your notes at least three times. Second and third time try to see how much you can remember from the first time.
• Go over the homework problems.
• Try to create your own problems similar to the ones I have given and solve them.
• Skim the relevant sections from the book.
• If you want to study in groups, at this point you are ready to quiz each other.

The practice midterm is below. Here are some facts about the actual midterm: (a) It is closed book, closed notes. Nothing will be permitted on your desk except your pen (pencil) and test. (b) You should bring photo ID. (c) There will be more than one version of the test. Each version will be of comparable difficulty. (d) One problem (less typos) on the actual test will be from the practice test.

1. Explain how to read in an image bob.png using Pillow, convert it to 1-byte grayscale, and then get the bytes into a 2D numpy array of the same dimensions as the original image.
2. Briefly define cross-validation and give one example exhaustive and one example inexhaustive cross-valdiation technique.
3. Prove `\hat{mu}_m = 1/m(sum_{i=1}^m x^{(i)})` is an unbiased estimator for the sample mean of a Gaussian distribution.
4. Explain the following concepts: (a) maximum likelihood estimator, (b) Kullback-Leibler Divergence, (c) cross-entropy.
5. Give the BFGS algorithm for minimizing a vector valued function.
6. Give some advantages and disadvantages of using sigmoid perceptrons versus ReLU versus leaky ReLU as our hidden units in a neural net.
7. Explain how `(\partial Loss(mathbb(y), vec{mathbb{x}}, \mathbf{W}))/(\partial W_i)` is computed for a neural network.
8. Give code to build a computation graph in TensorFlow of a single sigmoid perceptron unit with three input placeholders.
9. Write a python function leakyReluLayer(weights, inputs, biases, alpha) which will create a layer of TensorFlow Tensor object each compute leaky rectified linear units
10. What is a parameter norm penality? What is ridge regression? How does it compare with `L_1` - parameter regularization.